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VTOEO COMFORT NOISE ADDmON TEOamQlffi 

TECHNICAL FIELD 

This invention relates to a technique for reducing artifacts in connection with decoding 
of a coded video stream. 

BACKGROUND ART 

The decoding of a video stream compressed at low bit rate tends to cause visible 
artifacts noticeable to a viewer. Blockiness and structured noise patterns are common artifacts 
that arise when using block-based compression techniques. The human visual system has a 
greater sensitivity to certain types of artifacts, and thus, such artifacts appear more noticeable 
and objectionable than others. The addition of random noise to the decoded stream can 
reduce the noticeability of such compression artifacts, but large frame-to-frame differences 
created by adding random noise can itself produce artifacts that appear noticeable and 
objectionable. 

The addition of a dither signal can reduce human sensitivity to image artifacts, for 
example to hide contouring and blocking artifacts. One prior art technique has proposed 
adding a random noise dither that is based on film grain to an image to disguise block effects. 
The rationale for adding such random noise is that random error is more forgiving than the 
structure, or correlated error. Other prior art techniques have proposed adding a dither signal 
to a video stream to hide compression artifacts. Once past technique has proposed adding a 
random noise dither in the video encoding and decoding process in the in loop deblocking 
filter for the ITU/ISO H. 264 video coding standard, conmionly known as the JVT coding 
standard. The amount of dither to be added depends on the position of a pixel with respect to 
a block edge. Another prior technique has proposed adding that random noise subsequent to 
video decoding (i.e., adding noise as a '*post process")^ for use as comfort noise. The amount 
of noise added depends on the quantization parameter and on the amount of noise added to 
spatially neighboring pixels. The term "comfort noise" comes from the use of noise in audio 
compression to indicate noise pattern generated at the receiver end to avoid total silence that is 
uncomfortable to a listener. 
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Past techniques for reducing artifacts by adding noise typically reduce spatial artifacts 
at the risk of creating temporal abnormalities, i.e., large frame-to-frame differences. Thus, 
there exists a need for a technique for reducing artifacts during decoding of a coded video 
stieam that overcomies the aforementioned disadvantages. 

5 

BRIEF SUMMARY OF THE INVENTION 

Briefly, in accordance with a preferred embodiment of the present principles, there is 
provided a method for reducing artifacts in a video stream during decoding. The method 

10 conmiences by decoding the video stream. Following decoding, noise is added to the video 
stream by adding noise to each pixel in an amount correlated to the additive noise of pixels in 
a prior picture. Thus, in accordance with the present principles, temporal noise correlation 
aids in determining the additive noise to reduce large frame-to-£rame differences, a 
disadvantage of prior noise additive techniques. 

15 In accordance with another embodiment of the present principles, the prior picture 

from which the additive noise is derived comprises a displayed picture (i.e., a previously . 
decoded picture to which noise has been added). In yet another embodiment, the prior picture 
comprises a previously decoded picture. 

20 BRIEF DESCRIPTION OF THE DRAWINGS 

FIGURE 1 depicts a block schematic diagram of a video decoder in accordance with a 
first embodiment of the present principles, for reducing artifacts in connection with decoding 
a coded video stream; 

25 FIGURE 2 depicts a block schematic diagram of a video decoder in accordance with a 

second embodiment of the present principles, for reducing artifacts in connection with 
decoding a coded video stream; and 

FIGURE 3 depicts a block schematic diagram of a video decoder in accordance with a 
third embodiment of the present principles, for reducing artifacts in connection with decoding 

30 a coded video stream; 



wo 2005/020585 



PCT/US2004/025366 



-3- 

DETAILED DESCRIPTION 

In accordance with the present principles, adding a random-noise containing dither 
signal, to already decoded signal, in an amount correlated to the additive noise of pixels in a 
prior picture, serves to improve the subjective video quality. Figure 1 shows a block 
schematic diagram of a first decoder arrangement 10 for adding noise after decoding. The 
decoder arrangement 10 includes a decoder 12 for decoding an incoming coded video stream. 
The design of decoder 12 depends on the compression format employed to code the incoming 
video stream. In a preferred embodiment, the incoming video stream undergoes compression 
using the well-known ITU/ISO H. 264 standard. Under such circumstances, the decoder 12 
takes the form of a H.264 decoder known in the art. A reference picture store 14 stores 
pictures decoded by the decoder 12 for use by the decoder in decoding future pictures. 

A sununing block 18 sums each decoded picture firom the decoder 12 with noise from 
a noise generator 16. A clipper 20 then clips the resultant signal output by the summing block 
18 to yield a decoded picture for display which exhibits reduced artifacts. Note that noise 
addition occurs after storage of decoded pictures in the reference picture store 14 since the 
reference pictures must remain unchanged in order to properly decode the subsequent 
incoming pictures. 

The magnitude of the noise signal from the noise generator 16 added to each decoded 
picture typically depends on several different factors. To better understand the factors 
associated with noise addition, let the term N(k, x, y) represent the added pixel noise signal, 
P(k, X, y) represent the decoded pixel, and D (k, x, y) represent the displayed pixel (x, y) of 
the k* picture in the video sequence, respectively. The k^ pixel of the displayed picture 
becomes the sum of the decoded pixel plus the noise signal, as given by the relationship 

D(k,x,y) = aip(0,255,P(k,x,y) + N(k,x,y)) _ . (Equation 1) 

The visual impact of adding a noise signal to the video sequence, rather than just to a 
single image, becomes a consideration in the determination of the magnitude of the noise 
signal. In accordance with the present principles, the noise generator 16 correlates the 
magnitude of additive noise signal for at least one pixel in a picture to the value of the 
additive noise signal of at least one pixel in at least one previously displayed picture, i.e., a 
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decoded picture to which noise has been added. In an alternative embodiment of the current 
invention, the temporal correlation is based on the previously decoded picture (which contains 
no noise), rather than the previously displayed picture that contains noise. While noise is 
typically added to each pixel, comfort noise could be added to some pixels, not necessarily 
5 every pixel. 

fii first embodiment, noise addition using temporal correlation make use of a 
correlation factor a, 0 < a < 1, yielding the following relationship for added noise: 

N(k, X, y) = a N(k-1, x, y) + (1 ^ a) R(k, x, y) (Equation 2. ) 

10 

where R(k, x, y) is a random number generated using any type of random number distribution, 
for example a Gaussian or Laplacian distribution. Random nuniber generation can occur by 
means of a lookup table. The random number R(k, x, y) can also include spatial correlation, 
such as that used in film grain noise generation. 
15 The term a in Equation 2 has a value less than unity and to avoid division by a. 

Equation 2 can be simplified as follows: 

N(k,x,y) = (a*N(k-l.x,y) + (2**-^) R(k, x, y) +2'^^^>>b (Equations) 

20 with scaling variable b, chosen based on the desired precision of the division approximation, 
and the a =: round(a * 2^). 

FIGURE 2 depicts a block schematic diagram of a decoder arrangement 100 for 
adding temporally correlated noise to a video picture after decoding in accordance with a 

25 second illustrative embodiment of the present invention. The decoder arrangement 100 of 
HG. 2 contains elements in common with the decoder arrangement 10 of PIG. 1. Therefore, 
like numbers appear in FIG. 3 to identify like elements. The decoder arrangement 100 of FIG. 
2 includes a video decoder 12' which supplies a noise generator 16' with both a decoded 
picture, as well bit stream information contained in the decoded.picture. The bit stream 

30 information output by the decoder 12' can include a quantization parameter input to the noise 
generator. The severity of compression artifacts is correlated to the quantization parameter, 
with more severe compression artifacts occurring when high quantization parameter values 
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are used. The strength of the added comfort noise can be increased as the quantization 
parameter value increases. FIGURE 3 depicts a block schematic diagram of a decoder 
arrangement 1000 for adding temporally correlated noise to a video picture after decoding in 
accordance with a third illustrative embodiment of the present invention that is compatible 

5 with Equations 1 and 2. The decoder arrangement 1000 of FIG. 3 contains elements in 

connmon with the decoder arrangement 100 of FIG. 2. Therefore, like numbers appear in FIG, 
3 to identify like elements. Like the decoder arrangement 100 of FIG. 2, the decoder 
arrangement 1000 of FIG. 3 includes a video decoder 12' which supplies a noise generator 16' 
with both a decoded picture, as well bit stream information contained in the decoded picture. 

10 The decoder arrangement 1000 of FIG. 3 does differ from the decoder arrangement 100 of 
FIG. 2 in one respect. Unlike the decoder arrangement 100 of HG. 2, the decoder 
arrangenient 1000 of FIG. 3 includes a noise picture store 17 which stores the noise signal 
N(k, X, y) for the k* picture for subsequent use by the noise generator 16* . The decoder 
arrangement 1000 of FIGURE 3 requires more memory bandwidth than Uie decoder 

15 arrangements 10 and 100 of FIGS. 1 and 2, respectively. For each displayed picture, the 
decoder arrangement 1000 of FIG. 3 must access the previously displayed picture's noise 
signal and must also store the current picture's noise signal. 

Implementing Equation 3 in a decoder arrangement, such as the decoder arrangement 
1000 of Figure 3 will yields an instantiation of a temporal Infinite Impulse Response (IIR) 

20 filter. Such an DR filter can be generalized by using one or more filter taps. Such IIR filters 
can also generally be approximated using higher order FIR filters, using as many taps, t, as 
desired. 

N{k, Jc, y) = 2 ^'^^ ^^"^^ y> + " '(1- ^> ^ (^"^^ y> (Equation 4 ) 

(=0 

25 An FIR filter approach can be implemented using the decoder arrangement 100 of FIGURE 2. 
Only the previous random numbers R, rather than the previous noise, N, is used in an FIR 
filter approach, so memory bandwidth is reduced. 

The foregoing describes a technique for reducing artifacts in connection with decoding 
of a coded video stream by adding noise 



